Coding with asymmetric prior knowledge

We study the problem of compressed sensing with asymmetric prior information, as motivated by networking applications. We focus on the scenario in which a resource-limited encoder needs to report a small subset S from a universe of N objects to a more powerful decoder. The distinguishing feature of our model is asymmetry: the subset S is an i.i.d. sample from a prior distribution μ, and μ is only known to the decoder. This scenario implies that the encoder must use an oblivious compression scheme which can, nonetheless, achieve communication comparable to the entropy rate |S| ·H(μ), the standard benchmark when both encoder and decoder have access to the prior μ (as achieved by the Huffman code). In networks, this scenario models the omnipresent task of identity tracking: the encoder is a resource-limited tag in an Internet-ofThings universe, and is occasionally required to report a small subset of encountered tags to a more powerful gateway that resides closer to the datacenter. We first show that in order to exploit the prior μ in a non-trivial way in such asymmetric information scenario, the compression scheme must be randomized. This stands in contrast to the symmetric case (when both the encoder and decoder know μ), where the Huffman code provides a near-optimal deterministic solution. On the other hand, a rather simple argument shows that, when |S| = k, a random linear code achieves essentially optimal communication rate of O(k ·H(μ)) bits, nearly-matching the communication benchmark in the symmetric case. Alas, the resulting scheme has prohibitive decoding time: about ( N k ) ≈ (N/k). Our main result is a computationally efficient and linear coding scheme, which achieves an O(lg lgN)-competitive communication ratio compared to the optimal benchmark, and runs in poly(N, k) time. Our “multi-level” coding scheme uses a combination of hashing and syndromedecoding of Reed-Solomon codes, and relies on viewing the (unknown) prior μ as a rather small convex combination of uniform (“flat”) distributions.